A Speech Interface to Virtual Environments

Virtual reality has sometimes been thought of as embodying a return to a ‘natural’ way of interacting by direct manipulation of objects in a world. However, in the everyday world we also act through language: speaking is a ‘natural’ way of communicating our goals to others, and effecting changes in the world. In this paper, we discuss technical and design issues which need to be addressed in order to combine a direct manipulation interface to virtual reality with a speech interface. We then describe a prototype system based on intelligent agents which provide specialised functions in the virtual world. The agents have simple dialogue capabilities allowing users to directly control